Exploration via Model-based Interval Estimation

This paper takes an empirical approach to evaluating three model-based reinforcementlearning methods. All methods intend to speed the learning process by mixing exploitation of learned knowledge with exploration of possibly promising alternatives. We consider -greedy exploration, which is computationally cheap and popular, but unfocused in its exploration effort; R-Max exploration, a simplification of an exploration scheme that comes with a theoretical guarantee of efficiency; and a well-grounded approach, model-based interval estimation, that better integrates exploration and exploitation and achieves the best performance in our example tasks. Our experiments indicate that effective exploration can result in dramatic improvements in the observed rate of learning.